Method and apparatus for real-time control of physiological parameters

ABSTRACT

A real-time controller operating as an artificial pancreas uses a Kalman control algorithm to control glucose level of a patient in real time. The real-time controller receives an estimate of the patient glucose level and a reference glucose level. The estimate of the patient glucose level can be provided by an optimal estimator implemented using a linearized Kalman filter. The estimated glucose level and the reference glucose level are processed by the Kalman control algorithm to determine a control command in real time. The Kalman control algorithm has a dynamic process forced by the control command a cost function determining a relative level of control. The control command is provided to a dispenser which secretes insulin or glucagon in response to the control command to correct a relatively high glucose level or a relatively low glucose level.

PRIORITY CLAIM

[0001] This application is a continuation of U.S. application Ser. No.09/960,855, filed on Sep. 21, 2001, which claims the benefit of U.S.Provisional Application No. 60/234,632, filed on Sep. 22, 2000, andentitled Real Time Estimation & Control Of Biological Process.

COPYRIGHT RIGHTS

[0002] A portion of the disclosure of this patent document containsmaterial that is subject to copyright protection. The copyright ownerhas no objection to the facsimile reproduction by anyone of the patentdocument or of the patent disclosure as it appears in the Patent andTrademark Office patent files or records, but otherwise reserves allcopyright rights whatsoever.

BACKGROUND OF THE INVENTION

[0003] 1. Field of the Invention

[0004] This present invention relates generally to a method andapparatus for controlling physiological parameters and more particularlyto an optimal controller for controlling glucose levels in a patient.

[0005] 2. Description of the Related Art

[0006] Different types of sensors (e.g., optical sensors) are availablefor monitoring of physiological parameters (e.g., glucoseconcentration). Glucose monitoring is typically performed by people withdiabetes mellitus which is a medical condition involving a body'sinability to produce the quantity or quality of insulin needed tomaintain a normal circulating blood glucose. Frequent monitoring ofglucose is generally necessary to provide effective treatment and toprevent long term complications of diabetes (e.g., blindness, kidneyfailure, heart failure, etc.). New methods of monitoring glucose arefast, painless and convenient alternatives to the typical capillaryblood glucose (CBG) measurements which involve finger pricks that arepainful, inconvenient and difficult to perform for long term.

[0007] Optical measurement of glucose is performed by focusing a beam oflight onto the body. Optical sensors determine glucose concentration byanalyzing optical signal changes in wavelength, polarization orintensity of light. However, many factors other than glucoseconcentration also contribute to the optical signal changes. Forexample, sensor characteristics (e.g., aging), environmental variations(e.g., changes in temperature, humidity, skin hydration, pH, etc.), andphysiological variations (e.g., changes in tissue fluid due to activity,diet, medication or hormone fluctuations) affect sensor measurements.

[0008] Various methods are used to improve the accuracy of the sensormeasurements. One method (e.g., multivariate spectral analysis) utilizescalibration models developed by initially measuring known glucoseconcentrations to correct subsequent sensor measurements. Thecalibration models become inaccurate over time due to dynamic changes inphysiological processes. Another method (e.g., adaptive noise canceling)utilizes signal processing to cancel portions of the sensor measurementsunrelated to glucose concentration. For example, two substantiallysimultaneous sensor measurements at different wavelengths make up acomposite signal which can be processed to cancel its unknown anderratic portions. However, many sensors do not provide substantiallysimultaneous measurements at two different wavelengths.

SUMMARY OF THE INVENTION

[0009] The present invention solves these and other problems byproviding a method and apparatus for making optimal estimates of aphysiological parameter (e.g., glucose level), assessing reliability ofthe optimal estimates, and/or providing optimal control of thephysiological parameter in real time using one or more sensormeasurements at each measurement time epoch (or interval). The sensormeasurements can be time-based (e.g., every five minutes) to providecontinuous monitoring and/or regulation of the physiological parameter.The sensor measurements are a function of the physiological parameterwithin specified uncertainties.

[0010] An optimal estimator provides an accurate estimate of glucoselevel in real time using a sensor with at least one output. In oneembodiment, the optimal estimator is integrated with the sensor and anoutput display to be a compact glucose monitoring device which can beworn by a patient for continuous monitoring and real-time display ofglucose level. In an alternate embodiment, the optimal estimator is aseparate unit which can interface with different types of sensors andprovide one or more outputs for display, further processing by anotherdevice, or storage on a memory device.

[0011] In one embodiment, the optimal estimator employs a priorideterministic dynamic models developed with stochastic variables anduncertain parameters to make estimates of glucose level. For example,glucose level is defined as one of the stochastic (or random) variables.Dynamic mathematical models define process propagation (i.e., howphysiological and sensor parameters change in time) and measurementrelationship (i.e., how physiological and sensor parameters relate toenvironmental conditions). Environmental conditions (e.g., temperature,humidity, pH, patient activity, etc.) can be provided to the optimalestimator intermittently or periodically via environment sensors and/ordata entries by a patient or a doctor.

[0012] The optimal estimator uses dynamic models to propagate estimatesof respective stochastic variables, error variances, and errorcovariances forward in time. At each measurement time epoch, the optimalestimator generates real-time estimates of the stochastic variablesusing one or more sensor outputs and any ancillary input related toenvironmental conditions. In one embodiment, the optimal estimatoremploys a linearized Kalman filter to perform optimal estimation of thestochastic variables (e.g., glucose level). In particular, an extendedKalman filter is used to accommodate nonlinear stochastic models.

[0013] Before making real-time estimates, the optimal estimator isinitialized by providing initial values for the stochastic variables,error variances, and error covariances. For example, a CBG measurementor another direct glucose measurement is performed at initialization toprovide a starting value for the stochastic variable corresponding toglucose level.

[0014] In one embodiment, the optimal estimator provides one or moreoutputs to a patient health monitor which is capable of optimizedreal-time decisions and displays. The patient health monitor evaluatessystem performance by assessing the performance of the sensor and/oroptimal estimator in real time. For example, the patient health monitorapplies statistical testing to determine the reliability of thereal-time estimates of the stochastic variables by the optimalestimator. The statistical testing is performed in real time on residualerrors of the optimal estimator to establish performance measures.

[0015] In one embodiment, the patient health monitor acts as aninput/output interface between the patient or medical staff (e.g., adoctor, nurse, or other healthcare provider) and the optimal estimator.For example, environmental conditions can be provided to the patienthealth monitor for forwarding to the optimal estimator. Optimalestimator outputs can be provided to the patient health monitor fordisplay or forwarding to an external device (e.g., a computer or a datastorage device).

[0016] In one embodiment, the optimal estimator provides one or moreoutputs to an optimal controller which can regulate in real time thephysiological parameter being monitored. For example, an optimalcontroller responds to real-time optimal estimator outputs and providesan output to operate an actuator. In the case of glucose control, theactuator can be a dispenser or a pump which secretes insulin to correcta relatively high glucose level and glucagon to correct a relatively lowglucose level. The optimal controller takes advantage of a prioriinformation regarding the statistical characteristics of the actuatorand is able to control the output of the actuator to be within specifieduncertainties.

[0017] In one embodiment, the optimal estimator and the optimalcontroller form an optimal closed-loop system. For example, a glucosesensor, an optimal estimator, an optimal controller, and aninsulin/glucagon dispenser work together as an artificial pancreas tocontinuously regulate glucose level. The glucose sensor can be internalor external to a patient's body. The optimal controller provides acontrol feedback to the optimal estimator to account for delivery of theinsulin/glucagon.

[0018] The optimal closed-loop system is effective in a variety ofbiomedical applications. For example, cardiovascular functions can becontinuously regulated by using sensors to detect blood pressure, bloodoxygen level, physical activity and the like, an optimal estimator toprocess the sensor measurements and make real-time estimates of heartfunction parameters, and an optimal controller to control operations ofan artificial device (e.g., a pacemaker) in real time based on thereal-time estimates from the optimal estimator to achieve a set ofdesired heart function parameter values. Other artificial devices (e.g.,artificial limbs, bionic ears, and bionic eyes) can be part of similaroptimal closed-loop systems with sensors detecting nerve signals orother appropriate signals.

[0019] The optimal closed-loop system is also effective in optimaltreatment of chronic illnesses (e.g., HIV). Some medications fortreatment of chronic illnesses are relatively toxic to the body. Overdelivery of medication generally has adverse effects on the patient. Theoptimal closed-loop system is capable of providing effective and safetreatment for the patient. For example, an optimal estimator providesreal-time estimates of key physiological parameters using one or moresensors, and an optimal controller controls a slow infusion ofmedication in real time based on the real-time estimates from theoptimal estimator to obtain desirable values for the key physiologicalparameters.

[0020] In one embodiment, the optimal estimator, patient health monitor,and optimal controller are software algorithms which can be implementedusing respective microprocessors. New information regarding processpropagation or measurement relationship can be easily incorporated bymodifying, reconfiguring, and/or adding to the software algorithms. Theoptimal estimator, patient health monitor, and optimal controller can beimplemented as one joint algorithm or separate respective algorithmswhich function together to provide an optimal closed-loop system.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021]FIG. 1A is a block diagram of one embodiment of an estimator.

[0022]FIG. 1B is a block diagram of another embodiment of aglucose-monitoring device.

[0023]FIG. 2 illustrates one embodiment of modeling physiologicalprocesses in a linearized Kalman filter application.

[0024]FIG. 3A describes one embodiment of an estimation function whichdepicts a linearized Kalman filter formulation.

[0025]FIG. 3B illustrates an initialization in one embodiment of anoptimal estimator.

[0026]FIG. 3C illustrates a time-update cycle in accordance with oneembodiment of an optimal estimator.

[0027]FIG. 3D illustrates a measurement-update cycle in accordance withone embodiment of an optimal estimator.

[0028]FIGS. 4A and 4B illustrate a first set of time history plots ofoptimal glucose estimates and CBG measurements with respect to time.

[0029]FIGS. 5A and 5B illustrate a second set of time history plots ofoptimal glucose estimates and CBG measurements with respect to time.

[0030]FIG. 6A is a block diagram of one embodiment of a patient healthmonitor.

[0031]FIG. 6B illustrates one embodiment of a residual test data processin the patient health monitor.

[0032]FIG. 6C illustrates one embodiment of a statistical test processin the patient health monitor.

[0033]FIG. 6D illustrates one embodiment of an input/output interface inthe patient health monitor.

[0034]FIGS. 7A and 7B illustrate time history plots of residual testdata with respect to measurement time.

[0035]FIG. 8 is a functional diagram of one embodiment of an artificialpancreas.

[0036]FIG. 9A illustrates one embodiment of an optimal controller for aclosed loop system.

[0037]FIG. 9B illustrates one embodiment of a control model for acontroller in accordance with the present invention.

[0038]FIG. 9C illustrates one embodiment of a control algorithm.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0039] The present invention involves application of real-time optimalestimation, optimized real-time decisions and displays (e.g., a patienthealth monitor), or optimal real-time control to physiologicalprocesses. In one embodiment, the real-time optimal estimation, theoptimized real-time decisions and displays, and the optimal real-timecontrol are implemented as separate modules which can be combinedfunctionally. In an alternate embodiment, the real-time optimalestimation, the optimized real-time decisions and displays, and theoptimal real-time control are implemented as one joint algorithm.

[0040] In one embodiment, input to an optimal estimator is provided by aphysical sensor (or a plurality of sensors) which measures somearbitrary, but known, function (or functions) of variables or parametersto be estimated to within specified uncertainties and whose statisticalcharacteristics are known. In one embodiment, an output of a real-timecontroller is provided to a physical controllable dispenser, oractuator, whose output is some known function of parameters to becontrolled within specified uncertainties and whose statisticalcharacteristics are known. In one embodiment, a decision and displayfunction utilizes statistical testing of estimator residual errors usinginternally computed, and updated, estimator variances and covariances.

[0041] In one embodiment, the present invention is implemented as asoftware algorithm. The present invention uses models (e.g., dynamicprocess and measurement models). For best performance, the models shouldreflect the latest and most complete information available. As new andmore complete information is developed, performance can be improvedthrough incorporation of this information by simply modifying thesoftware algorithm of the present invention.

[0042] Embodiments of the present invention will be describedhereinafter with reference to the drawings. FIG. 1A is a block diagramof one embodiment of an estimator. The estimator uses a linearizedKalman filter. The linearized Kalman filter accommodates nonlinearprocess models and/or nonlinear measurement models. In one embodiment,the linearized Kalman filter is a discrete extended Kalman filter whichis linearized after each update using best estimates.

[0043] A general formulation of a continuous-discrete extended Kalmanfilter is provided in Table 1. In an alternate embodiment, thelinearized Kalman filter is linearized about a nominal set for which ageneral formulation is given in Table 2.

[0044] The estimator computes an estimator gain based on time updatedand measurement updated error variable variances and covariances. In oneembodiment, the estimator is implemented using discrete formulations. Inan alternate embodiment, the estimator is implemented using continuousformulations. In the actual development of algorithms, one can choose aCovariance formulation or an Information formulation depending oninitialization uncertainty considerations. Further, the use of Biermanfactorization techniques (UDU{circumflex over ( )}T) in theimplementation leads to numerically stable algorithms which areexcellent for operation over very long time periods.

[0045] In one embodiment, the estimator of FIG. 1A is applied to theproblem of monitoring patient glucose levels. Any type of physicalsensor which measures some function of glucose can be used. In oneembodiment, one or more capillary blood glucose (CBG) measurements areobtained on initialization of the estimator and when estimated glucosedeviations exceed computed variance levels.

[0046] In another embodiment, additional capillary glucose values areobtained on a periodic basis to assess sensor function. As an example,CBG measurements may be obtained once or twice a day pre and one hourpost prandial or when the estimator determines that the glucose valuesare out of range of predetermined limits.

[0047] In one embodiment, the estimator is contained in a small portablepackage. The estimator can be operated by a patient. Alternatively, theestimator can be operated by medical staff in a hospital or clinic. Theestimator (e.g., Kalman estimator) uses a Kalman filter to providedreal-time estimates of patient glucose levels based upon glucose sensormeasurements. The Kalman estimator can also use additional informationto more closely predict time propagation of glucose levels, such asexercise, food intake, insulin administration, or other factors whichinfluence glucose levels or the function of the sensor (e.g., local pH,temperature or oxygen tension near the physical sensor).

[0048]FIG. 1B is a block diagram of one embodiment of aglucose-monitoring device 100. The glucose-monitoring device 100includes a glucose sensor 108, an ancillary sensor 110, a glucoseestimator 114, and a patient health monitor 114. The glucose-monitoringdevice 100 is used to provide real-time glucose estimates of a patient102. In one embodiment, the glucose-monitoring device 100 is anintegrated unit which is portable by the patient 102 to providecontinuous glucose monitoring and real-time displays.

[0049] The glucose sensor 108 (e.g., a probe, patch, infrared, or lasersensor) is coupled to the patient 102 and outputs a measurement ƒ(g)which is a function of the glucose level of the patient 102. The glucosesensor 108 makes measurements within specified uncertainties and hasknown statistical characteristics. The glucose sensor 108 provides themeasurement ƒ(g) to the glucose estimator 112. In one embodiment, theglucose sensor 108 makes measurements periodically. In an alternateembodiment, the glucose sensor 108 makes measurements intermittently orupon command.

[0050] The ancillary sensor 110 is coupled to the patient 102, theglucose sensor 108, and/or surroundings of the patient 102 and/orglucose sensor 108 to provide information regarding environmental and/orglucose sensor conditions (e.g., temperature, humidity, local pH, etc.)which affect the measurement. One or more outputs of the ancillarysensor 110 are provided to the glucose estimator 112. In one embodiment,the ancillary sensor 110 provides outputs to the patient health monitor114 which can process the information for display or for forwarding tothe glucose estimator 112. In an alternate embodiment, the ancillarysensor 110 is not a part of the glucose-monitoring device 100.

[0051] The patient 102 and/or a medical staff 104 (i.e., a user) canprovide information on the environmental and glucose sensor conditionsas well as other information affecting the measurement. In oneembodiment, the patient 102 and/or medical staff 104 inputs information(e.g., exercise activity, food intake, insulin administration, etc.)using the patient health monitor 114. The patient health monitor 114acts as an input/output interface or a means for the user to configurethe glucose estimator 112. The patient health monitor 114 forwards theinformation to the glucose estimator 112.

[0052] In one embodiment, the patient health monitor 114 is a displaydevice. For example, an output of the glucose estimator 112 (e.g., anoptimal real-time estimate of glucose) is provided to the patient healthmonitor 114 which displays the information in a comprehensible formatfor the patient 102 and/or the medical staff 104. The patient healthmonitor 114 can also contemporaneously display information provided bythe glucose sensor 108, the ancillary sensor 110, the patient 102 and/orthe medical staff 104 which affects the measurement used to make theoptimal glucose estimate.

[0053] In another embodiment, the patient health monitor 114 is a statusindicator. For example, the glucose estimator 112 provides outputs(e.g., residuals and variances) to the patient health monitor 114 whichapplies statistical testing to determine the reliability of the glucosesensor 108 and/or the glucose estimator 112. The patient health monitor114 provides a warning when poor performance is detected.

[0054] In one embodiment, the glucose estimator 112 is a linearizedKalman filter (e.g., a discrete extended Kalman filter) to account for anonlinear process model and/or measurement model. The glucose estimator(or Kalman estimator) 112 provides real-time estimates of the glucoselevel in the patient 102. An initialization measurement 106 (e.g., a CBGmeasurement) is obtained from the patient 102 and provided to the Kalmanestimator 112 to initialize the Kalman estimator 112.

[0055]FIG. 2 is a flow chart illustrating one embodiment of a modelingprocess for physiological processes. In one embodiment, physiologicalprocesses are described by nonlinear stochastic models. In oneembodiment, the flow chart of FIG. 2 illustrates one method ofdeveloping a dynamic model for an optimal estimator. The method includessteps for state vector development, nominal dynamic/measurement modeldevelopment, linearized model development, and uncertainty modeling andnominal model verification.

[0056]FIG. 2 illustrates modeling of physiological processes in alinearized Kalman filter application. As an example in glucoseestimation, in a first block of FIG. 2, glucose is an estimationvariable. The time rate of change of glucose might become anotherestimation variable. An uncertain parameter might be the glucose sensorscale factor, and additionally, the rate of change of scale factor overtime could be another.

[0057] In one embodiment of a second block of FIG. 2, the way in whichvariables and parameters nominally propagate over time may change withconditions. In the example of glucose estimation, inputs can be used toidentify patient related activities: eating; exercising; sleeping;insulin injection; etc. With these identified patient relatedactivities, additional state variables can be identified and modeled.For example, if patient eating can be related to a change in glucoselevel over some specified time interval, dynamics can be implementedwithin the estimator model which will propagate (or extrapolate) a risein the glucose level over that time interval. This rise may be modeledby appropriate functions whose variables contain uncertainties, whichmay be added as elements of the state vector. In an analogous way,decreases in glucose levels (e.g., due to insulin injections) can bemodeled. These models may be general in nature, or they may be patientspecific. Consequently, patient related activities that have asignificant impact on glucose levels, or the rate of change of glucoselevels, can be accounted for within the dynamic process model.

[0058] Insofar as sensor modeling is concerned, experience shows that,for example, infrared sensor measurement bias errors vary with, amongother things, temperature. If this variation with temperature can bemodeled, and included in the process model, then a temperaturemeasurement will improve estimator performance. A particular physicalglucose sensor may have a scale factor which has a characteristic decayin sensitivity over time as discussed in further detail below. There maybe other variables that can be measured which will affect the physicalfunction of a sensor such as local pH, oxygen tension, etc.

[0059] In one embodiment, the nominal modeling is comprised of threetypes: 1) predictable characteristics of a physical sensor function overtime (e.g., a fixed rate in decline of sensor output), 2) othermeasurable physical variables which may affect sensor function (e.g.,local temperature, pH, etc.), and 3) predictable changes in the modelwhich occur with patient related activities (e.g., exercise, eating orinsulin administration). Changes to the dynamic process model may addvariables and/or uncertain parameters to the state vector and changes tomeasurement models. As a result, the activities indicated in blocks 1and 2 of FIG. 2 constitute an iterative process.

[0060] In one embodiment of block 3 of FIG. 2, the relationship betweenvariables, parameters, and measurements determines which parts of theprocesses and measurements are nonlinear and are linearized. Further,the relationship between variables, parameters, and measurementsdetermines if the variables and parameters are observable and can beestimated. In certain cases, observability can be enhanced throughintroduction of additional modeling information. For the glucoseexample, the ability to estimate sensor scale factor and/or detectsensor failure may be improved by modeling glucose propagation changesdue to insulin injections or ingestion of sugar. For example, trackingknown changes enhances scale factor observability through estimatorgenerated correlations. Further, if it is known that glucose levels varyand the sensor measurement does not change accordingly within prescribedlevels of uncertainty, a sensor problem is indicated.

[0061] In one embodiment of block 4 of FIG. 2, the development of arelatively large database is used to empirically verify and/or modifythe nominal nonlinear dynamic process/measurement models and deriveuncertainty levels associated with the variables, parameters, andmeasurements. In one embodiment, the empirical data can be fitted tononlinear functions using a nonlinear regression package contained in acommercially available software application program such as“Mathematica” from Wolfram Research, Inc. Analytical functions may beadded or modified using the test database. The repeatability of the fitover nominal ranges of the patient environment determines the uncertainparameters and the variations establish uncertainty levels. The moreaccurate the dynamic process and measurement models, the more theuncertainties are reduced, and the greater the estimator performance. Asthe process evolves, the modeling becomes better defined throughiterations between blocks 4 and 2. Certain portions of the models may bedeveloped on an individual basis (e.g., for a specific patient).

[0062] A database is used to empirically develop and verify models. Anembodiment discussed below uses the database to develop two separatedynamic process models. For example, after processing a number of datasets from a physical sensor that was used to monitor the glucose levelof various patients, it was observed that the sensor scale factor wasequally likely to move up or down over the first fifteen to twenty-fivehours. However, the scale factor tended to decay for the remaining lifeof the sensor after this period of time, usually three or four days.These observations are incorporated in an embodiment of the estimatordiscussed in further detail below.

[0063]FIG. 3A describes an estimation function which depicts alinearized Kalman filter formulation. In one embodiment, the particularform is that of a discrete extended Kalman filter which linearizes aftereach update using best estimates. In the formulation, a vector whoseelements comprise variables and/or parameters (with uncertainties) formaking estimates defines the state of a system.

[0064] There is a distinction between variables (e.g., random variables)and parameters with uncertainty. Variables, such as glucose and rate ofchange of glucose, are estimated and can be controlled (if control isimplemented). Parameters with uncertainty are part of the modelstructure not known precisely and are estimated and updated (likevariables) but not controlled, e.g., a glucose sensor scale factor orinsulin dispensing controller scale factor.

[0065] In one embodiment, real-time variable and parameter estimates areused to re-linearize the model following each update. Inputs to theestimator can consist of any measurement which is related to, or can becorrelated with, any element in the state vector. In the case of glucoseestimation, other inputs can consist of dynamic process configurationcontrol based on patient related activities or other circumstances.Following initialization, the time update and measurement update cyclesform a recursive loop. The time update period is the time intervalbetween the receipt of measurements. This is a function of the sensorand of acceptable latency in the estimates.

[0066]FIGS. 3B, 3C and 3D illustrate, respectively, Initialization andthe recursive Time Update and Measurement Update cycles according to oneembodiment of the present invention. Table 3 defines, in more detail,symbols used in these figures. In one embodiment, dynamic process andmeasurement modeling is contained in software algorithms with parameterand structure updates in real time.

[0067] One embodiment of an initialization of the estimation process isdescribed in FIG. 3B. A state vector estimate, Xeo, contains the initialestimates of the process variables and model parameters withuncertainty, while the covariance matrix, Po, contains the initialvariances and covariances associated with the Xeo elements.

[0068]FIG. 3B shows an example embodiment with a two-element statevector which is based on empirical observations described above. As anexample, the point in time when the scale factor begins to decay waschosen as 20 hours, a nominal value over the database. An exponentialdecay was chosen to model this decay rate and is consistent with thefirst derivative of scale factor equal to a parameter, alpha, multipliedby the variable scale factor. In this case, alpha is not modeled as anuncertain parameter; a value of 0.012 was chosen as a nominal value,over the database, for the five-minute cycle time of this physicalsensor.

[0069] The initial estimate of glucose is 150 mg/dl (state vectorelement) with an initial uncertainty variance of 100 mg/dl squared (acovariance matrix element). When processing the data, the glucoseelement was initialized by setting it substantially equal to the firstcapillary blood glucose measurement. The nominal initial scale factorvalue, for the physical sensor, is 0.25 nano-amps/(mg/dl) with aparameter uncertainty (variance) of 0.1 nano-amps/(mg/dl) squared.

[0070] The initial covariance between the glucose variable and the scalefactor parameter is zero. Correlation between glucose and scale factorwill develop as the estimator processes the sensor measurements. Eachtype of sensor will have its own model and characteristics. Themeasurement uncertainty is 5 nano-amps squared, an element of the Rmatrix. In one embodiment, this is a scalar measurement and the R matrixcontains a single element.

[0071] A second measurement in this example embodiment is an occasionaldirect measurement, such as a capillary blood glucose measurement (CBG),with a unity scale factor, and a measurement error uncertainty of 15mg/dl squared (a single element in a second R matrix). This is probablybetter modeled as 15% of the measured glucose value. The growth inuncertainty of glucose and scale factor from measurement to measurementis, respectively, 20 mg/dl squared and 0.002 nano-amps squared (elementsof the process noise matrix, Q).

[0072]FIG. 3C illustrates a time-update process of the recursive processin one embodiment of an optimal estimator. In this figure, a negativesuperscript indicates a time update while a positive superscriptindicates a measurement update. In the brackets, a letter “i” indicatestime at the ith interval and (i−1) indicates time at the previous timeinterval. In an update process, a state vector is first updated sincethese elements are used to update matrices and to bring the time epochof the estimated measurement to be consistent with that of the nextmeasurement to be received.

[0073] In an example embodiment with a two-element state vector, thedynamic process is linear. With no patient inputs, the first derivativeof the measured physiologic variable is zero, corresponding to the casewhen the level (on average) is as likely to either go up or to go down.No additional a priori information is assumed about the time propagationof glucose.

[0074] In one embodiment of estimating glucose, the solution to thescale factor propagation after the first 20 hours is defined by theexponential shown in the second column. Consequently, for thisembodiment, the dynamic process function (f), is linear, is not afunction of state vector elements, and, from linear system theory, thetransition matrix (A) is the 2 by 2 identity matrix for the first 20hours and thereafter is defined by the 2 by 2 matrix in the secondcolumn.

[0075] In one embodiment, the measurement function (h) for the sensormeasurement is non-linear in the state vector elements. If Ge and Se areused to denote glucose and scale factor estimates, respectively, thenYe=Se*Ge. Definitions of the above terms are provided in Table 3. Whenlinearized using best estimates, the linearized measurement matrix H=[SeGe] and is of the same functional form both before and after 20 hours.

[0076]FIG. 3D illustrates a measurement-update process of one embodimentof an optimal estimator. The measurement update sequence begins with thecomputation of the gain matrix, K(i). The difference between the actualsensor measurement and the best estimate of the measurement is computed:y(i)=Ym(i)−Se(i)*Ge(i). This difference, or residual, when multiplied bythe gain matrix and added to the time-updated estimate produces themeasurement-updated estimate.

[0077] The covariance matrix is then measurement updated, reflecting thelevel of uncertainty in the estimates following the processing of ameasurement. In the glucose example, if a second measurement isavailable, such as a CBG, then the measurement sequence is again cycledthrough, starting with a new gain computation, and using the appropriatemeasurement matrix and new best estimate of the next measurement.Following the processing of all available measurements at the ith timeepoch, the updated state vector and covariance matrix are then used tostart the time update for the (i+1)th time epoch which begins the nextcycle.

[0078] One embodiment of an estimation algorithm illustrating theinitialization, the time update process, and the measurement updateprocess discussed above is provided in Table 4. Table 4 is an algorithmprogrammed in the MATLAB language (from Math Works). This printoutdefines a working program and has been used to process a significantnumber of data sets. In the glucose example, the estimation results fromthe process of two data files, both gathered from the same patient, andtaken about a month apart, are shown in FIGS. 4A, 4B, 5A, and 5B.

[0079] In one embodiment, sensor inputs are provided and processed every5 minutes. Occasional CBGs are also provided. For example, two CBGs perday were processed by an estimator; and additional CBG values were usedto judge estimator performance by comparing glucose estimates withactual CBG values not used by the estimator.

[0080]FIGS. 4A and 4B illustrate a first set of time history plots ofoptimal glucose estimates and CBG measurements with respect to time.FIG. 4A shows a time history of the real-time estimates of glucose(e.g., every 5 minutes) along with all available discrete CBGs.

[0081]FIG. 4B shows a time history of the glucose estimates along withCBGs that were processed. In this figure, the estimated glucose valuetook several rapid swings between approximately 48 hours and 58 hours.FIG. 4B indicates that only sensor measurements were processed duringthat interval and no CBGs were processed. However, the CBGs plotted inFIG. 4A indicate that the glucose estimates did tend to follow theexcursions of the patient glucose levels.

[0082]FIGS. 5A and 5B illustrate a second set of time history plots ofoptimal glucose estimates and CBG measurements with respect to time. Thetime duration for these runs is about 4 days without patient inputs.Over this time period, the patient ate, slept, exercised, and tookinsulin injections. Dynamic models to account for these activities, muchlike the decaying scale factor, could be implemented and called into useupon command.

[0083]FIG. 6A is a block diagram of one embodiment of a patient healthmonitor. On example of an algorithm for the patient health monitor is inTable 4 which generates statistical test data based on Kalman filterresiduals as well as test displays. In one embodiment, the patienthealth monitor generates real-time decisions and displays which areintegrated with a Kalman filter. The patient health monitor allows thepatient or medical staff to interact with a Kalman estimator and/orKalman controller described herein.

[0084] In one embodiment, the patient health monitor provides insightinto how well a Kalman filter is working through the filter residualthat is the difference between the estimate of the measurement at thetime the measurement is received and the actual measurement. In anotherembodiment, other checks are used from time to time, such as the CBGs inthe glucose example described above. An example of another check on thestatus of a sensor is through the use of patient inputs or signalsindicating that something is changing in a prescribed way and thennoting whether or not the sensor is observing this change withinprescribed uncertainties.

[0085] In one embodiment, the real-time displays and decisions of thepatient health monitor uses some occasional outside checks but reliessubstantially on results of statistical testing performed on filterresiduals. If estimates of the measurements are, on average, good (e.g.,residuals are small and unbiased), then the estimator is generallyworking well, and vice versa. More specifically, elements of thecovariance matrix can be used to construct statistical testapplications.

[0086] For hospital applications, real-time displays of glucoseestimates along with real-time displays of estimator performance testresults can be important visual inputs to the medical staff. Requestsfor additional CBG measurements or the sounding of an alarm in the eventglucose estimates exceed critical limits may also prove useful. In oneembodiment, a reduced number of outputs is provided in relatively smallestimators for individual use.

[0087]FIG. 6B illustrates one embodiment of a residual test data processin the patient health monitor. For example, the residual (y), covariancematrix (P), measurement matrix (H), and measurement noise matrix (R) areavailable from an estimator algorithm at each time epoch (i). If theestimator is operating properly, the sequence of residuals has theproperty of zero mean, white noise sequence, i.e., any two residualstaken at different times are uncorrelated (E[y(i)*y(j)]=0, for all j notequal to i). This condition provides a unique means for constructingstatistical tests.

[0088] Visually, a time history plot of the residuals, Sy(i) in FIG. 6B,should appear random, zero mean, and unbaised. If they are summed overtime, the deviation of the sum from zero should, on average, grow as thesquare root of time, as should its absolute value, ASy(i). The fact thatE[y(i)*y(j)]=0, for i not equal to j, also means that the sum of thevariances, SV(i) in FIG. 6B, which is easily computed, is equal to thevariance of the sum of residuals for a properly performing estimator.The standard deviation of the sum of residuals, StdSV(i) also grows asthe square root of time.

[0089]FIG. 6C illustrates one embodiment of a statistical test processin the patient health monitor. FIG. 6C defines statistical tests whichcan be constructed based on filter residual test data. One of thesecompares the absolute value of the sum of the residuals with thestandard deviation of the sum of residuals. On average, the ASy(i)should be bounded by the StdSV(i). If not, this indicates that thedeviations of Sy(i) are growing faster than that of a white noisesequence, implying degraded estimator performance.

[0090] Real-time displays of ASy(i) and StdSV(i) histories can provide avisual picture of estimator performance. An example is provided in FIGS.7A and 7B which illustrate time history plots of residual test data withrespect to measurement time. These two plots are one form of residualtests for the two sets of glucose estimation results provided in FIGS.4A, 4B, 5A, and 5B respectively. In both cases, the sum of residuals arewell behaved and was bounded by the Standard Deviation, Std, of the sumof residuals.

[0091] The example embodiment for the glucose application of test data,test generation, and test result display described above is implementedby the algorithm in Table 4. Other real-time quantitative tests can alsobe constructed using these data. Tests on individual residuals can beperformed using individual variances. If a measurement is received whichcauses the residual to exceed a four signal level, for example, theaction might be to emit a warning and request an immediate CBGmeasurement. Other tests are identified in FIG. 6C and will be evaluatedas the system develops.

[0092]FIG. 6D illustrates one embodiment of an input/output interface inthe patient health monitor. The patient health monitor provides acapability for the patient and/or staff to communicate and interact inreal time with estimation and control processes using simple commands,visual displays, and audio outputs described above. Confidence levels inestimator performance can be established and a genuine interfaceestablished whereby the estimation and control processes could requestadditional information to check and insure confidence in estimates,physical sensors, and physical controllers. The staff can provide usefulreal-time inputs to augment this process.

[0093]FIG. 8 is a block diagram of one embodiment of a Kalman optimalstochastic control solution as applied to physiological processes. Inone embodiment, the application includes both optimal stochasticregulator and optimal stochastic tracking control solutions. Trackingcontrol involves a controlled variable following a reference value,constant or dynamic, as closely as possible. Control is applied tophysiological processes, wherein a control gain is computed based uponoptimization criteria which minimizes controlled variable errors whileminimizing application of control based on cost weightings.

[0094] There is a duality between the computation of a Kalman estimatorgain and a Kalman control gain. The Kalman estimator gain minimizesestimation error variances. The Kalman control gain minimizes variancesof error between the controlled variables and the reference variableswhile minimizing the level of control applied. As a result, the optimalcontrol function includes specification of the controlled variables andtheir associated costs as well as costs associated with the amount ofcontrol to be applied. For example, smaller control variable error costsand larger control application costs will allow the controlled variableto deviate farther from the reference, but with reduced application ofcontrol.

[0095] In one embodiment, linearization techniques described above inassociation with the Kalman estimator is applied to physiologicalnonlinear stochastic processes. For example, linearization about nominalvalues or about best estimates are provided by the Kalman estimator.Uncertain parameters associated with a controllable dispenser, oractuator, are included in an estimator state vector.

[0096] The optimal stochastic controller can use linear or nonlinearformulations and discrete or continuous time formulations. In oneembodiment, the optimal stochastic controller is used with an optimalestimator described herein and/or an optimized decision and displayfunction also described herein to form a closed loop system. The closedloop system works as an artificial pancreas when applied to a glucoseproblem in one embodiment.

[0097]FIG. 8 is a functional diagram showing a controllable dispenser,or pump, with the capability to secrete insulin and glucagon to controlhigh and low glucose levels, respectively. In one embodiment of a closedloop system, an estimator and a controller share a state vector whereinthe estimator estimates it and the controller controls designatedelements of it. Unlike the estimator, the time varying gain computationsfor the controller is computationally intensive and may not be used inall applications. The controller includes time varying or steady stategain formulations.

[0098]FIG. 9A illustrates one embodiment of an optimal controller forthe artificial pancreas. The glucose control problem is a trackingproblem since the glucose level is controlled to a desired level whichmay either be constant or a function of time. In one embodiment, thedynamic process model includes a quantitative description of how glucoselevels propagate, in time, as a function of insulin/glucagon secreted bythe controllable pump. The dynamic process model is described by asystem of first order differential or difference equations. Excludingthe pump, much of the modeling for the glucose control is availablethrough the estimator modeling development.

[0099]FIG. 9B illustrates one embodiment of a control model for acontroller. In an example embodiment, the dynamic process model isdescribed by three first order differential equations that are forced bya control variable, u. The dynamics of this process include a firstorder time lag and a scale factor associated with the pump, and a firstorder time lag and a scale factor associated with the physiologicalprocess. The model for the glucose sensors is the same as in the exampleglucose estimator embodiment. In the definition of the elements of thetransition matrix, alpha g, s, and d are the inverse of the first ordertime lags associated with glucose, glucose sensor scale factor, andpump, respectively. Beta g is the glucose scale factor multiplied byalpha g. Delta t is the time interval between measurement/controlapplication epochs. The controlled error is the difference between theestimate of glucose (provided by the estimator) and the glucose controlpoint input.

[0100] In a controller command, Beta d is a pump scale factor multipliedby alpha d. The measurements are the same as for the example glucoseestimator, except that the measurements contain an additional zero sincethe measurements are not functions of the variable insulin/glucagon.

[0101] The cost function, which is minimized by the optimal stochasticcontrol, contains costs associated with the glucose error and theapplication of control. Choosing a value of Cu that is much larger thanCg will result in a relative gentle application of control. Anotherexample embodiment would utilize a higher order model in which the firstderivative of glucose would be included in the state vector, andincluded as a control variable. If a relatively large cost is associatedwith the first derivative relative to the glucose control point error,then the control will be very active when rapid changes in patientglucose occur while relatively gentle otherwise. Other exampleembodiments would include pump scale factor as a state vector element(uncertain parameter) as well as insulin/glucagon measurements to theestimator.

[0102]FIG. 9C illustrates one embodiment of a control algorithm based onthe model defined in FIG. 9B. Using either a time varying gain or apre-computed steady state gain, control is applied at each epoch basedupon a difference between an estimate of a patient glucose level and aglucose control point. A control, u, is applied to a pump over each timeinterval, and a state vector is time updated as indicated in this Figureusing the control variable, u, as a forcing function. Other time updatesare performed in accordance with embodiments discussed above inassociation with estimator equations.

[0103] A potential problem in the application of closed loop control tothat of physiological processes is due to potentially long time delaysthat may be de-stabilizing. These delays can be in the form of transportlags. A transport lag is the time between when control is applied andwhen the process action begins. An optimal technique for control usingdelay states is discussed in “Optimal Control of Linear StochasticSystems with Process and Observation Time Delays” by E. J. Knobbe(Academic Press, Inc., 1989) and is hereby incorporated herein in itsentirety by reference thereto. A discussion of principles developed foroptimal control with process and observation time delays is provided inTable 5.

[0104] Although described above in connection with particularembodiments of the present invention, it should be understood that thedescriptions of the embodiments are illustrative of the invention andare not intended to be limiting. Various modifications and applicationsmay occur to those skilled in the art without departing from the truespirit and scope of the invention. TABLE 1 SUMMARY OFCONTINUOUS-DISCRETE EXTENDED KALMAN FILTER System Model${{\overset{.}{\underset{\_}{x}}(t)} = {{\underset{\_}{f}\left( {{\underset{\_}{x}(t)},t} \right)} + {\underset{\_}{w}(t)}}};{\left. {\underset{\_}{w}(t)} \right.\sim{N\left( {\underset{\_}{0},{Q(t)}} \right)}}$

Measurement Model${{{\underset{\_}{z}}_{k} = {{{\underset{\_}{h}}_{k}\left( {\underset{\_}{x}\left( t_{\underset{\_}{k}} \right)} \right)} + {\underset{\_}{v}}_{k}}};\quad {k = 1}},2,\quad {\ldots \quad;\quad {\left. {\underset{\_}{v}}_{k} \right.\sim{N\left( {\underset{\_}{0},R_{k}} \right)}}}$

Initial Conditions$\left. {\underset{\_}{x}(0)} \right.\sim{N\left( {{\hat{\underset{\_}{x}}}_{o},P_{o}} \right)}$

Other Assumptions${E\left\lbrack {{\underset{\_}{w}(t)}v_{k}^{T}} \right\rbrack} = {0\quad {for}\quad {all}\quad k\quad {and}\quad {all}\quad t}$

State Estimate Propagation${\overset{.}{\hat{\underset{\_}{x}}}(t)} = {\underset{\_}{f}\left( {{\hat{\underset{\_}{x}}(t)},t} \right)}$

Error Covariance Propagation${\overset{.}{P}(t)} = {{{\hat{F}\left( {{\hat{\underset{\_}{x}}(t)},t} \right)}{P(t)}} + {{P(t)}{F^{T}\left( {{\hat{\underset{\_}{x}}(t)},t} \right)}} + {Q(t)}}$

State Estimate Update${{\hat{\underset{\_}{x}}}_{k}( + )} = {{{\hat{\underset{\_}{x}}}_{k}( - )} + {K_{k}\left\lbrack {{\underset{\_}{z}}_{k} - {{\underset{\_}{h}}_{k}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}} \right\rbrack}}$

Error Covariance Update${P_{k}( + )} = {\left\lbrack {I - {K_{k}{H_{k}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}}} \right\rbrack {P_{k}( - )}}$

Gain Matrix$K_{k} = {{P_{k}( - )}{{H_{k}^{T}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}\left\lbrack {{{H_{k}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}\quad {P_{k}( - )}{H_{k}^{T}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}} + R_{k}} \right\rbrack}^{- 1}}$

Definitions $\begin{matrix}{{{F\left( {{\hat{\underset{\_}{x}}(t)},t} \right)} = \frac{\partial{\underset{\_}{f}\left( {{\underset{\_}{x}(t)},t} \right)}}{\partial{\underset{\_}{x}(t)}}}}_{{\underset{\_}{x}{(t)}} = {\hat{\underset{\_}{x}}{(t)}}} \\{{{H_{k}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)} = \frac{\partial{{\underset{\_}{h}}_{k}\left( {\underset{\_}{x}\left( t_{k} \right)} \right)}}{\partial{\underset{\_}{x}\left( t_{k} \right)}}}}_{{\underset{\_}{x}{(t_{k})}} = {{\hat{\underset{\_}{x}}}_{k}{( - )}}}\end{matrix}\quad$

[0105] TABLE 2 SUMMARY OF CONTINUOUS-DISCRETE LINEARIZED KALMAN FILTERSystem Model${{\overset{.}{\underset{\_}{x}}(t)} = {{\underset{\_}{f}\left( {{\underset{\_}{x}(t)},t} \right)} + {\underset{\_}{w}(t)}}};{\left. {\underset{\_}{w}(t)} \right.\sim{N\left( {\underset{\_}{0},{Q(t)}} \right)}}$

Measurement Model${{{\underset{\_}{z}}_{k} = {{{\underset{\_}{h}}_{k}\left( {\underset{\_}{x}\left( t_{\underset{\_}{k}} \right)} \right)} + {\underset{\_}{v}}_{k}}};\quad {k = 1}},2,\quad {\ldots \quad;\quad {\left. {\underset{\_}{v}}_{k} \right.\sim{N\left( {\underset{\_}{0},R_{k}} \right)}}}$

Initial Conditions$\left. {\underset{\_}{x}(0)} \right.\sim{N\left( {{\hat{\underset{\_}{x}}}_{o},P_{o}} \right)}$

Other Assumptions $\begin{matrix}{{E\left\lbrack {{\underset{\_}{w}(t)}v_{k}^{T}} \right\rbrack} = {0\quad {for}\quad {all}\quad k\quad {and}\quad {all}\quad t}} \\{{Nominal}\quad {trajectory}\quad {\overset{\_}{\underset{\_}{x}}(t)}\quad {is}\quad {available}}\end{matrix}\quad$

State Estimate Propagation${\overset{.}{\hat{\underset{\_}{x}}}(t)} = {{\underset{\_}{f}\left( {{\overset{\_}{\underset{\_}{x}}(t)},t} \right)} + {{P\left( {{\overset{\_}{\underset{\_}{x}}(t)},t} \right)}\left\lbrack {{\overset{\_}{\underset{\_}{x}}(t)} - {\overset{\_}{\underset{\_}{x}}(t)}} \right\rbrack}}$

Error Covariance Propagation${\overset{.}{P}(t)} = {{{\hat{F}\left( {{\hat{\underset{\_}{x}}(t)},t} \right)}{P(t)}} + {{P(t)}{F^{T}\left( {{\hat{\underset{\_}{x}}(t)},t} \right)}} + {Q(t)}}$

State Estimate Update${{\hat{\underset{\_}{x}}}_{k}( + )} = {{{\hat{\underset{\_}{x}}}_{k}( - )} + {K_{k}\left\lbrack {{\underset{\_}{z}}_{k} - {{\underset{\_}{b}}_{k}\left( {\overset{\_}{\underset{\_}{x}}\left( t_{k} \right)} \right)} - {{H_{k}\left( {\overset{\_}{\underset{\_}{x}}(t)} \right)}\left\lbrack {{{\hat{\underset{\_}{x}}}_{k}( - )} - {\overset{\_}{\underset{\_}{x}}\left( t_{k} \right)}} \right\rbrack}} \right\rbrack}}$

Error Covariance Update${P_{k}( + )} = {\left\lbrack {I - {K_{k}{H_{k}\left( {\hat{\underset{\_}{x}}\left( t_{k} \right)} \right)}}} \right\rbrack {P_{k}( - )}}$

Gain Matrix$K_{k} = {{P_{k}( - )}{{H_{k}^{T}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}\left\lbrack {{{H_{k}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}\quad {P_{k}( - )}{H_{k}^{T}\left( {{\hat{\underset{\_}{x}}}_{k}( - )} \right)}} + R_{k}} \right\rbrack}^{- 1}}$

Definitions $\begin{matrix}{{{F\left( {{\hat{\underset{\_}{x}}(t)},t} \right)} = \frac{\partial{\underset{\_}{f}\left( {{\underset{\_}{x}(t)},t} \right)}}{\partial{\underset{\_}{x}(t)}}}}_{{\underset{\_}{x}{(t)}} = {\overset{\_}{\underset{\_}{x}}{(t)}}} \\{{{H_{k}\left( {{\overset{\_}{\underset{\_}{x}}}_{k}\left( t_{k} \right)} \right)} = \frac{\partial{{\underset{\_}{h}}_{k}\left( {\underset{\_}{x}\left( t_{k} \right)} \right)}}{\partial{\underset{\_}{x}\left( t_{k} \right)}}}}_{{\underset{\_}{x}{(t_{k})}} = {{\overset{\_}{\underset{\_}{x}}}_{k}{(t_{k})}}}\end{matrix}\quad$

[0106] TABLE 3 State Vector X, Xe, Xe(0): These symbols denote,respectively, the state vector, the best estimate (or optimal estimate)of the state vector, and the initial best estimate of the state vector.They contain both variables and parameters as defined above and, withregard to estimation, there are no distinctions. Dynamic Process ModeldX = f(X,t), dXe = f(Xe,t): These symbols denote the time derivative ofthe state vector and estimated state vector, respectively. Theyrepresent a system of first order differential equations (or differenceequations) which describe the manner in which the state elementspropagate in time. The function, f, may be linear or nonlinear.Measurement Model Ym = h(X) + e, Ye = h(Xe): These symbols denote theactual measurement and the best estimate of the measurement,respectively. The function, h, defines the arbitrary, but known, way inwhich the state vector elements are related to the measurement, and erepresents the sensor measurement error. Covariance Matrix P, P(0):Denotes, respectively, the matrix of variances and co-variancesassociated with the errors of each of the state variable estimates(within the estimated state vector) and their initial values, TransitionMatrix A, A(0): Denotes the transition matrix, which is used topropagate the covariance matrix forward it time (time update), and it'sinitial value, respectively; in the linear case it is also used to timeupdate the state vector estimate. Process Noise Matrix Q: Denotes thematrix of variances and co-variances associated with error growthuncertainty accumulated in the state variable estimates since the lastmeasurement update, Measurement Matrix H, H(0): Denotes the measurementmatrix and it's initial value, respectively. This matrix defines thelinear functional relationship between the measurement and the statevector elements. If, the measurement model is linear, then H = h; if his non-linear, then H is defined by linearizing h using partialderivatives or perturbation techniques. Measurement Noise Matrix R:Denotes the matrix of variances and co-variances associated withmeasurement error uncertainties. Kalman Gain Matrix K: Denotes theKalman gain matrix which, when multiplied by the difference between theactual measurement, and the best estimate of the measurement, yields theestimated state correction. This estimated state correction, when addedto the old best estimate, becomes the new best estimate. EstimatedMeasurement Error (Residual) y = Ym − Ye: Denotes the difference betweenthe actual measurement and the best estimate of the measurement. Thisdifference multiplied by the Kalman gain yields the correction to theprevious state vector estimate.

[0107] TABLE 4 % Optimal Glucose Estimator. Extended Kalman Filter;Computes Statistical Test Data for Patient Health Monitor%Ed_JYP_Estimator; Two-step scale-factor model, Exp = .999 for t>20;r=15{circumflex over ( )}2 for y<10 ; x=[150;.25]; hh=[10];p=[100{circumflex over ( )}2 0 ;0 .1{circumflex over ( )}2];a=[1 0 ;0 1]; q=[20{circumflex over ( )}2 0 ; 0 .002{circumflex over ( )}2];rr=15{circumflex over ( )}2; r= 5{circumflex over ( )}2; I=[1 0;0 1];load EdJY528.prn, [m,n]=size(EdJY528), t=EdJY528 1:, 1)/12;y=EdJY528(:,2); g=EdJY528(:,3); gm=EdJYS28(:,4); x=[gmf(1); .25];sz=0.0; SVZ = p(1,1)*K(2){circumflex over ( )}2 + p(2,2)*x(1){circumflexover ( )}2; for i = 1:m; . . . r = 5{circumflex over ( )}2; . . . zz =0; . . . if y(i) < 10 r = 15{circumflex over ( )}2; . . . end; if t(i) >20 a =(1 0 ; 0 .999]; . . . end; p=a*p*a′+q; . . . x=a*x;. . . ifgm(i) > 0 k=p*hh*/(hh*p*hh*+rr); . . . zz=(gm(i)−x(1)); . . . x=x+k*zz;. . . p=(I−k*hh) *p; . . . end; h=[x(2) x(1)]; . . . k=p*h′/ (h*p*hh+r);. . . x=x+k*z; . . . p=(I−k*h) *p; . . . az=abs (z); . . . sz=sz+z;. . .asz = abs (sz); . . . VZ = p(1,1) *x(2){circumflex over ( )}2 +p(2,2)*x(1){circumflex over ( )}2 + 2*p(1,2)*x(1)*x(2)+ r ; . . . sigZ =sqrt(VZ); . . . SVZ = SVZ +VZ; . . . siqSZ = sqrt(SVZ); . . . VZZ =p(1.1) + rr; . . . sigZZ = sqrt(VZZ); . . . azz = abs(zz); . . .xhistory(i,1) = x(1); . . . xhistory(i,2) = x(2); . . . gmhistory =gm; .. . ghistory =g; . . . zhistory(i) =z; . . . sigsz_history(i) = sigSZ; .. . sigz_history(i) = 2*sigZ; . . . aszhistory(i) = asz; . . .azhistory(i) = az; . . . zzhistory(i) = zz; . . . sigzz_history(i) =2*sigZZ; . . . azzhistory(i) = azz; . . . sig_history(i,1)=sqrt(p(1,1));. . . sig_history(i,2)=sqrt(p(2,2)); . . . yhistory=y;. . . end; figure(1) plot(t,xhistory(:,2)); xlabel (′ Measurement Time (hrs) ′); ylabel(′Scale Factor Estimate; (ISIG Units/(mg/dl))′); title(′Scale FactorEstimate vs Time′); figure (2) plot (t, gmhistory (:), t,xhistory(:,1)); xlabel(′Time (hrs)′); Ylabel(′Glucose Meas & Estimated Glucose;(mg/dl)′); title(′All Glucose Measurements & Estimated Glucose vsTime′); figure (3) plot (t,yhistory (:), t, zhistory(:)); xlabel(′Time(hrs)′); Ylabel(′ISIG Meas & z Residual; (ISIG Units) ′); title(′ISIGMeasurement & z residual vs Time′); figure (4)plot(t,xhistory(:,1),t,ghistory(:,1)); xlabel(′ Time (hrs)′);ylabel(′Glucose & Estimated Glucose; (mg/dl)′); title(′Glucose Estimate& All Glucose CBGs vs Time′); figure (5)plot(t,sigzz_history(:),t,azzhistory(:)); xlabel (′Time (hrs) ′);ylabel(′Abs(zz) Residual & two sigma(zz); (mg/dl) ′); title(′ResidualTest: 2 Siqma(zz) & Abs(zz) Residual vs Time′); figure (6)plot(t,azhistory(:),t,sigz_history(:)); xlabel(′Measurement Time(hrs)′); ylabel(′z Residual & Standard Deviation; (ISIG Units)′);title(′Residual Test Abs(z) & 2 Sigma(z) vs Time′); figure (7)plot(t,aszhistory(:),t,sigsz_history(:)); xlabel(′ Measurement Time(hrs)′); ylabel(′ sz Residual & Sigma; (ISIG Units) ′); title(′ResidualTest : Abs(sz) & Sigma(sz) vs Time′);

[0108] TABLE 5 PROBLEM DEFINITION The general, discrete-time systemrepresentation with explicit process and observation time delays isgiven by${{x\left( {k + 1} \right)} = {{\sum\limits_{i = 1}^{q}{{A_{i}(k)}{X\left( {k + 1 - i} \right)}}} + {{B(k)}{U(k)}} + {W_{1}(k)}}},$

(1)${{Y(k)} = {{\sum\limits_{j = 1}^{p}{{C_{j}(k)}{X\left( {k + 1 - j} \right)}}} + {W_{2}(k)}}},$

(2) where p and q are integers > 1; W₁ and W₂ are zero-mean white-noisesequences such that ${{\left. \begin{matrix}{{E\left\{ {{W_{1}(k)}{W_{1}^{T}(k)}} \right\}} = {V_{1}(k)}} \\{{E\left\{ {{W_{2}(k)}{W_{2}^{T}(k)}} \right\}} = {V_{2}(k)}} \\{{E\left\{ {{W_{1}(i)}{W_{1}^{T}(j)}} \right\}} = 0}\end{matrix} \right\} \quad \forall_{i,j,k}};}\quad$

Y is an m × 1 observation vector; X is an n × 1 random state vectorwhose initial uncertainty is uncorrelated with W₁ and W₂ and withinitial covariance Q₀; and U is the control input vector. The objectiveis to find the control function (functional) U(k) for k = 1, 2, . . .that minimizes an expected quadratic cost function for the linearstochastic regulator defined by Eqs. (1) and (2). Because linearstochastic tracking problems can be formulated as linear stochasticregulator problems by combining the reference and plant models in anaugmented system [3], this system representation applies equally to thetracking problem. PROBLEM FORMULATION AND SOLUTION The systemrepresentation defined by Eqs. (1) and (2) can be cast in stochasticregulator form by augmenting the state vector with the time-delayedstates, that is,${{\overset{\_}{X}\left( {k + 1} \right)} = {{{\overset{\_}{A}(k)}{\overset{\_}{X}(k)}} + {{\overset{\_}{B}(k)}^{*}{U(k)}} + {{\overset{\_}{W}}_{1}(k)}}},$

(3) and${Y(k)} = {{{\overset{\_}{C}(k)}{\overset{\_}{X}(k)}} + {W_{2}(k)}}$

(4) $\begin{matrix}{{{\overset{\_}{A}(k)}\overset{\Delta}{=}\begin{bmatrix}0 & I & 0 & \cdots & 0 \\\quad & 0 & \quad & \ddots & \quad \\\vdots & \quad & \ddots & \quad & \vdots \\\quad & \ddots & \quad & \quad & \quad \\0 & \quad & \cdots & 0 & I \\{A_{k}(k)} & \quad & \cdots & \quad & {A_{1}(k)}\end{bmatrix}},{{\overset{\_}{X}(k)}\overset{\Delta}{=}\begin{bmatrix}{X\left( {k - h + 1} \right)} \\{X\left( {k - h + 2} \right)} \\\vdots \\{X\left( {k - 1} \right)} \\{X(k)}\end{bmatrix}},} \\{{{\overset{\_}{B}(k)}\overset{\Delta}{=}\begin{bmatrix}0 \\\vdots \\0 \\{B(k)}\end{bmatrix}},{{\overset{\_}{W}(k)}\overset{\Delta}{=}\begin{bmatrix}0 \\\vdots \\0 \\{W_{1}(k)}\end{bmatrix}},{{\overset{\_}{C}(k)}^{T}\overset{\Delta}{=}\begin{bmatrix}{C_{h}(k)} \\{C_{h - 1}(k)} \\\vdots \\{C_{2}(k)} \\{C_{1}(k)}\end{bmatrix}},}\end{matrix}\quad$

and the dimensionality of the system is defined by h, where h = max(p,q) In Eq. (3), the control remains the same as in Eq. (1) because;physically, past states cannot be changed or controlled. In Eq. (4) theobservation Y and the observation noise W₂ also remain unchanged. InEqs. (1) and (2), p and q are not, in general. equal. For example. ifthe observation is not a linear function of all the delayed statescontained in {overscore (X)}. then h = q > p. and the appropriate (q −p) submatrices in {overscore (C)}. of Eq. (4). are set equal to zero. Ifthe process evolution is not a linear function of all the delayed statescontained in {overscore (X)}, then h = p > q, and the appropriatesubmatrices in {overscore (A)}. of Eq. (3). are set equal to zero. Thedimensions of {overscore (B)} and {overscore (W)}₁,in Eq. (3), must alsobe consistent with the integer h. Therefore, without loss of generality,the system defined by Eqs. (1) and (2) can be represented by theaugmented system model described by Eqs. (3) and (4). Given that thecontrol is to minimize the expected value of a quadratic cost functionof the form,${E\left\{ {{\sum\limits_{k = k_{0}}^{k_{1} - 1}\left\lbrack {{{\overset{\_}{X}(k)}^{T}{{\overset{\_}{R}}_{1}(k)}{\overset{\_}{X}(k)}} + {{U^{T}(k)}{R_{2}(k)}{U(k)}}} \right\rbrack} + {{\overset{\_}{X}\left( k_{1} \right)}{\overset{\_}{P}\left( k_{1} \right)}{\overset{\_}{X}\left( k_{1} \right)}}} \right\}},$

(5) then the separation principle applies [3]. Also, the optimal linearstochastic control of the augmented system is given by a deterministic,optimal linear controller with state input {overscore (X)} (or estimatedstate feedback), which is provided by an optimal one-step predictorusing the augmented model [4]. That is,${{{U(k)} = {{- {\overset{\_}{F}(k)}}{\hat{\overset{\_}{X}}(k)}}};\quad {k = k_{0}}},{k_{0} + 1},\ldots,k_{1},$

(6) where the control gain {overscore (F)} satisfies $\begin{matrix}{{\overset{\_}{F}(k)} = \left\{ {{R_{2}(k)} + {{{\overset{\_}{B}}^{T}(k)}\left\lbrack {{{\overset{\_}{R}}_{1}\left( {k + 1} \right)} + {\overset{\_}{P}\left( {k +} \right.}} \right.}} \right.} \\{{\left. {\left. 1 \right\rbrack {B(k)}} \right\}^{- 1}{{B^{T}(k)}\left\lbrack {{\overset{\_}{R}\left( {k + 1} \right)} + {P\left( {k + 1} \right)}} \right\rbrack}{\overset{\_}{A}(k)}};}\end{matrix}\quad$

(7) the matrix {overscore (P)} satisfies the recursive matrix Riccatiequation,${\overset{\_}{P}(k)} = {{{\overset{\_}{A}}^{1}(k)}\left\lbrack {{{{\overset{\_}{R}}_{1}\left( {k + 1} \right)} + {{\overset{\_}{P}\left( {k + 1} \right\rbrack}\left\lbrack {{\overset{\_}{A}(k)} - {{\overset{\_}{B}(k)}{\overset{\_}{F}(k)}}} \right\rbrack}};} \right.}$

(8)${the}\quad {one}\text{-}{step}\quad {predictor}\quad {output}\quad \hat{\overset{\_}{X}}\quad {is}\quad {defined}\quad {by}$

${{\hat{\overset{\_}{X}}\left( {k + 1} \right)} = {{{\overset{\_}{A}(k)}{\hat{\overset{\_}{X}}(k)}} + {{\overset{\_}{B}(k)}{U(k)}} + {{\overset{\_}{K}(k)}\left\lbrack {{Y(k)} - {{\overset{\_}{C}(k)}{\hat{\overset{\_}{X}}(k)}}} \right\rbrack}}};$

(9) the estimator gain {overscore (K)} satisfies${{\overset{\_}{K}(k)} = {{\overset{\_}{A}(k)}{\overset{\_}{Q}(k)}{{{\overset{\_}{C}}^{T}(k)}\left\lbrack {{{\overset{\_}{C}(k)}{\overset{\_}{Q}(k)}{{\overset{\_}{C}}^{T}(k)}} + {V_{2}(k)}} \right\rbrack}^{- 1}}};$

(10) and the state estimation error covariance matrix {overscore (Q)}satisfies the recursive matrix Riccati equation,${\overset{\_}{Q}\left( {k + 1} \right)} = {{\left\lbrack {{\overset{\_}{A}(k)} - {{\overset{\_}{K}(k)}{\overset{\_}{C}(k)}}} \right\rbrack {\overset{\_}{Q}(k)}{{\overset{\_}{A}}^{T}(k)}} + {{{\overset{\_}{V}}_{1}(k)}.}}$

(11) The final value of {overscore (P)} used to “initialize” Eq. (8)(which is solved backward in time) is the final value defined in thequadratic cost function of Eq. (5), that is, {overscore (P)}(k₁) ={overscore (P)}₁. The initial value of {overscore (Q)} used toinitialize Eq. (11) is the error covariance of the initial estimate ofX, that is, {overscore (Q)}(0) = {overscore (Q)}₀. If the systemstatistics defined in Eqs. (1) and (2) are gaussian, then the abovesolution is the optimal solution without qualification; if not, then itis the optimal linear control solution. The expected system performanceis determined by analyzing the augmented system as a linear, stochasticregulator problem PRACTICAL APPLICATIONS From control gain Eqs. (7) and(8) and estimator gain Eqs. (10) and (11), it can be shown that therequired dimensions of the controller and estimator are not, in general,equal. The dimensions of the control matrix Riccati equation (8) isdetermined by the number of delayed states in the process evolution,that is, {overscore (P)} has the dimensions of n · q × n · q.Consequently, for the special case of measurement delays only (q = 1),the controller implementation is unaffected. However, the dimensions ofthe estimator matrix Riccati equation (11) are determined by the maximumof p and q. Since h = max(p, q), then {overscore (Q)} has the dimensionsn · h × n · h. For the general case, the optimal control is given by${{U(k)} = {- {\sum\limits_{i = 1}^{q}{{F_{i}(k)}{\hat{X}\left( {k + 1 - i} \right)}}}}},$

(12) where ${{\overset{\_}{F}(k)}\overset{\Delta}{=}\begin{bmatrix}{F_{q}(k)} & {F_{q - 1}(k)} & \ldots & {F_{2}(k)} & {F_{1}(k)}\end{bmatrix}},$

(13) and${{\hat{\overset{\_}{X}}}^{T}(k)}\overset{\Delta}{=}{\begin{bmatrix}{{\hat{X}}^{T}\left( {k + 1 - h} \right)} & \ldots & {{\hat{X}}^{T}\left( {k + 1 - q} \right)} & \ldots & {{\hat{X}}^{T}\left( {k - 1} \right)} & {{\hat{X}}^{T}(k)}\end{bmatrix}.}$

(14) We note that, in Eq. (14), {circumflex over (X)}(k) is the one-steppredicted value of the original system state vector; {circumflex over(X)}(k − 1) is the filtered value; {circumflex over (X)}(k − 2) is theone-step smoothed value; and, finally, {circumflex over (X)}(k + 1 − h)is the (h − 2)th smoothed value. Typically, the real-time computationalrequirements associated with the implementation of time-varying optimalcontrol are always stressing due to the “backward-in-time” recursionthat is required to obtain solutions {overscore (P)} of Eq. (8).However, in many high-accuracy applications, the {overscore (A)},{overscore (B)}, {overscore (R)}₁, and R₂ matrices of the augmentedsystem and cost function can be treated as time invariant over the timeintervals of interest. Further, if the augmented system satisfies therelatively minor requirements of stabilizability and detectability, thenthe control gain {overscore (F)} will converge to a unique value suchthat the steady- state optimal control law is time invariant,asymptotically stable, and mini- mizes the quadratic cost function of Eq(5) as k₁ → ∞. For this case, the steady-state gain matrix {overscore(F)}_(ss) can be computed off-line and stored for real- time use; thereal-time computations required to implement this steady-state controllaw are negligible. Usually, for tracking and regulator problems, thesteady-state control gains and time-varying gains are such that theinitial value of the time-varying gain is equal to the steady-state gainbut is less than or equal to the steady-state gain as time progresses,that is,${\overset{\_}{F}}_{ss} \geq {{\overset{\_}{F}(k)}\quad {for}\quad k_{o}} \leq k \leq {k_{1}.}$

As a consequence, the steady-state gain tends to maintain the controlledstate closer to the estimated state, but at the cost of control energy.If accuracy is the significant criterion, then the steady-state gain isnot only easy to implement but also provides essentially equivalent orbetter accuracy. This, however, is not the case for the estimator gain,even though in most cases where linear steady-state optimal stochasticcontrol is implemented, both the steady-state control and the estimatorgains are used. For the es- timator, and the same general conditions asbefore, the initial time-varying estimator gain is usually significantlylarger than the steady-state gain, to account for initial uncertaintiesin the knowledge of the system state. As time progresses, thetime-varying gain converges to the steady-state gain, that is,${\overset{\_}{K}}_{ss} \leq {{\overset{\_}{K}(k)}\quad {for}\quad k_{o}} \leq k \leq {k_{1}.}$

Consequently, initial system performance is significantly degraded ifK_(ss) is used. In fact, if applied to a “linearized” system, theestimator can actually diverge given the initial small steady-statefilter gains. Hence a good compromise between performance andcomputational complexity is to choose the steady-state controller withthe time-varying estimator. Further, because the estimator matrixRiccati equation is solved forward in time, the computations associatedwith the time-varying filter gain are orders of magnitude less than withthe time-varying control gain and can usually be implemented in realtime. Although the computations associated with the augmented system es-timator are significantly increased because of the increased dimensions,some simplifications can be made. The estimator Riccati equation can beseparated into a time update and a measurement update, where the timeupdate for the augmented system becomes primarily one of data transfer.If {overscore (Q)}, from Eq. (11), is defined as${{{Q(k)} = \begin{bmatrix}{Q_{1,1}(k)} & {Q_{1,2}(k)} & \cdots & {Q_{1,h}(k)} \\{Q_{2,1}(k)} & {Q_{2,2}(k)} & \cdots & \vdots \\\vdots & \quad & \ddots & \quad \\{Q_{h,1}(k)} & \cdots & \quad & {Q_{h,h}(k)}\end{bmatrix}},}\quad$

(15) then, for the special case where q = 1 and p ≧ 2, the time updatebecomes $\left. {\begin{matrix}{{\left. \begin{matrix}{{Q_{i,j}\left( {k + 1} \right)} = {Q_{{i + 1},{j + 1}}(k)}} \\{{Q_{i,P}\left( {k + 1} \right)} = {{Q_{{i + 1},P}(k)}{A_{1}^{T}(k)}}}\end{matrix} \right\} \quad i},{j = 1},\ldots \quad,{p - 1}} \\{{Q_{p,p}\left( {k + 1} \right)} = {{{A_{1}(k)}{Q_{p,p}(k)}{A_{1}^{T}(k)}} + {V_{1}(k)}}}\end{matrix}\quad} \right\}.$

(16) For the worst case, where q > p > I, the time update is given by$\left. \begin{matrix}{{{Q_{i,j}\left( {k + 1} \right)} = {Q_{{i + 1},{j + 1}}(k)}},\quad i,{j = 1},\ldots \quad,{q - 1},} \\{{{Q_{i,q}\left( {k + 1} \right)} = {\sum\limits_{j = 1}^{q}{{Q_{i,j}(k)}{A_{{({q + 1})} - j}^{T}(k)}}}},\quad {i = 1},\ldots \quad,{q - 1},} \\{{Q_{q,q}\left( {k + 1} \right)} = {{\sum\limits_{i = 1}^{q}{\sum\limits_{j = 1}^{q}{{A_{{({q + 1})} - i}(k)}{Q_{i,j}(k)}{A_{{({q + 1})} - j}^{T}(k)}}}} + {V_{1}(k)}}}\end{matrix} \right\}.$

(17) In practice, another computational simplification results becauserarely, if ever, is the actual system measurement a function of everytime-delayed state element, and likewise for the actual system processmodel. Hence the submatrices A_(i) and C_(i) (for i > 1) in {overscore(A)} and {overscore (C)}, respectively, are usually of significantlyreduced dimension. This, in turn, significantly reduces the dimensionsof the augmented system model. We note that the iteration interval Δt(implied in the discrete-time system representation) or some integernumber of Δt's should be set equal to the time delay. For variable timedelays, it may be advantageous to use (a) variable iteration intervalsfor the estimator, (b) fixed iteration intervals for the controller, and(c) variable-time updates of the state estimate to time synch theestimator output with the controller input. Finally, for the analogouscontinuous-time problem, the filter equation becomes a partialdifferential equation with a boundary condition, and the covananceequation becomes a partial differential matrix equation with threeboundary conditions (see [2]). Consequently, the most practical controlsolution is obtained by discretizing the continuous-time systemrepresentation and then applying the approach of Section III. Onetechnique for discretizing the continuous-time representation is byusing the Z transform method; see, for example, [5]. Using a scalardifferential equation with one delayed slate as an example, we have${\overset{.}{X}(t)} = {{\sum\limits_{i = 0}^{1}{a_{i}{X\left( {t - {i\quad {\Delta t}}} \right)}}} + {{U(t)}.}}$

(18) If we assume that the output solution X(t) is sampled in discretetime, and if the system process can be reasonably approximated by acontinuous system in which {dot over (X)} is driven by the output of ahigh- frequency sampling of the right side of Eq. (18), then thediscrete-time process model is given byX(k + 1) = (1 + a₀Δt)X(k) + (a₁Δt)X(k − Δt) + (Δt)U(k).

(19) The approximate discrete-time solution to Eq. (18), which is givenby Eq. (19), is now in a form consistent with Eq. (1) of Section II, andthe ap- proach of Section III can be applied.

What is claimed is:
 1. A method for close-loop control of a physiological parameter comprising the acts of: obtaining a measurement of the physiological parameter from a patient; providing the measurement to an optimal estimator in real time, wherein the optimal estimator outputs a best estimate of the physiological parameter in real time based on the measurement; providing the best estimate of the physiological parameter to an optimal controller in real time, wherein the optimal controller outputs a control command in real time based on the best estimate of the physiological parameter and a control reference; and providing the control command to an actuator, wherein the actuator provides an output to adjust the physiological parameter.
 2. The method of claim 1, wherein the measurement is obtained using a sensor.
 3. The method of claim 1, wherein the optimal estimator is implemented using a linearized Kalman algorithm.
 4. The method of claim 1, wherein the control reference is provided by a patient health monitor. 